The present invention relates to the field of optical imagers. More particularly, the invention relates to a method for adaptively expanding the dynamic range of optical active pixel sensors, by using on-chip, real-time automatic scaling of each pixel.
Optical imagers (sensors) are widely used in many imaging applications, such as metrology, avionics and space, and particularly as light sensing devices in electronic cameras. Charge Coupled Devices (CCDs) are widely used in optical imagers, and have 65 to 75 dB dynamic range. However, CCDs require special processing, and also lack itegrability. An emerging technology is currently exploited for manufacturing Metal Oxide Semiconductor (CMOS) Active Pixel Sensors (APS), which consume less power and are potentially of lower cost and high integrability.
A typical Active Pixel Sensor (APS) consists of an array of active pixels, each active pixel containing a light sensing element (e.g., a photo-diode or a photo-transistor) and one or more transistors to buffer and amplify the electric signals generated by the light sensing element, in response to light exposure.
Basically, optical imagers may be exposed to a wide range of illumination levels when imaging scenes. Night vision usually introduces illumination levels starting from 10xe2x88x923 lux, whereas indoor lighting ranges between 102 and 103 lux, and sunlight may reach 105 lux. This wide range requires a wide dynamic range from the sensors employed.
The dynamic range of a pixel is defined as 20*log(S/N), where S and N are the saturation level of the pixel, and the noise floor, respectively. Low dynamic range entails saturation of pixels with high sensitivity, in case of high illumination levels, or high noise levels, of pixels with lower sensitivity. In both cases, part of the information is lost. Typical dynamic ranges of an APS also range between 65 to 75 dB.
An important factor affecting the dynamic range is the pixel integration time (the time lapses between reset and sample signal), during which the pixel is exposed to illumination and outputs a corresponding electric signal. Basically, a relatively long integration time is required for low illumination levels, so as to obtain a signal which is well beyond the noise floor. On the other hand, a relatively short integration time is required for high illumination levels, so as to eliminate saturation.
Several known methods are used for widening the dynamic range of an APS, which fall into three basic categories: compressing the pixel response curve, clipping the pixel response curve and controlling the pixel integration time. The first two are less advantageous, since they result in loss of information. The latter is preferred, and can be done either globally or locally.
Israeli Patent 100620 describes a method for increasing the dynamic range of optical sensors by conditionally applying a chain of reset signals, within the frame time. A control circuit, which may be common to a group of pixels, compares the illumination levels to a threshold level, which indicates impending saturation, and enables a reset signal after the threshold has been reached. The number of resets during the frame time is counted, and used to calculate a scaling factor, by which the output electric signal is multiplied. However, the circuitry which is required to carry out the comparisons at different time points, which is described in this patent, occupies relatively large area. Hence, the fill-factor (which is the ratio between the pixel area which is responsive to light, and the total pixel area) of the imager is reduced, causing a deterioration in the resolution of the imager.
WO 97/17800 describes an imaging device, comprising an array of pixels having two sampling capacitor banks. Each row is sampled and copied to each capacitor bank twice, first for the short integration time and second, for the long integration time, thereby widening the dynamic range. However, this method is advantageous only if the actual illumination level matches one of these integration times. Any illumination level that falls in between, results in a loss of information. Furthermore, storing the outputs for additional integration times requires more memory cells, which occupy more space, thereby reducing the Field Of View (FOV).
All the methods described above have not yet provided satisfactory solutions to the task of expanding the dynamic range of optical imagers, in real time, and without losing information.
It is an object of the present invention to provide a method and apparatus for expanding the dynamic range of optical imagers, which overcome the drawbacks of prior art imagers.
It is another object of the invention to provide a method for expanding the dynamic range of optical imagers, in real time and during the frame time.
It is still another object of the invention to provide a method for expanding the dynamic range of optical imagers, without losing temporal resolution of the imager, and with minimal effect on spatial resolution.
It is yet another object of the invention to provide a method for expanding the dynamic range of optical imagers, that matches any illumination level.
Other purposes and advantages of the invention will appear as the description proceeds.
The invention is directed to a method for expanding the dynamic range of an optical imager by individually controlling the integration time of each pixel of a sensor array, and providing a corresponding scaling factor for the electrical output of each individual pixel during the frame time. The integration time of each pixel is controlled as a function of light intensity received thereon, by resetting the pixel after a predetermined threshold for the output signal has been reached. The imager is constructed from a two dimensional active pixel array of M (integer) columns and N (integer) rows, fabricated on a semiconductor substrate. Each individual pixel contains an optical sensor to receive light, a reset input and an electrical output representing the illumination received thereon. The outputs of a selected row are copied into an upper column-parallel signal chain of M capacitors and compared to a set of corresponding threshold values, and into a lower column-parallel signal chain of M capacitors for readout. Preferably, the electrical readout of each pixel is output as an analog signal, or converted to a digital representation. The comparison results are stored in a digital memory. A control circuit controls the reading operations of each pixel, the timing of comparisons for each pixel, and provides corresponding reset signals for each pixel. The required expansion of the dynamic range is determined by a series of W bits, and comparisons are carried out at W time points, having prefixed intervals. Preferably, these time points are selected according to the downgoing series       T    -          T              U        1              ,      T    -          T              U        2              ,  …  ⁢      xe2x80x83    ,      T    -          T              U        W              ,
wherein U greater than 1 and T represents the full integration time. Time and space are multiplexed by matching between each time point and a row which is shifted from a selected row n, which is selected for comparison, by a prefixed number of rows. Preferably, row shifts are selected according to the integer values of the downgoing series       n    -          N              U        1              ,      n    -          N              U        2              ,  …  ⁢      xe2x80x83    ,      n    -                  N                  U          W                    .      
Preferably, U is selected to be 2, so as to simplify the matching between       T    -          T              2        1              ,      T    -          T              2        2              ,  …  ⁢      xe2x80x83    ,      T    -          T              2        W            
and       n    -          N              2        1              ,      n    -          N              2        2              ,  …  ⁢      xe2x80x83    ,      n    -          N              2        W              ,
required for time-space multiplexing. Preferably, the downgoing series of row shifts may be also selected as the integer values of the series       N          X      1        ,      N          X      2        ,  …  ⁢      xe2x80x83    ,      N          X      W      
where Xi greater than 1 and i=1, 2, . . . , W. Preferably, the downgoing series of time points may be       T    -          T              X        1              ,      T    -          T              X        2              ,  …  ⁢      xe2x80x83    ,      T    -                  T                  X          W                    .      
The comparison results are stored in a memory, and a reset pulse is applied for those pixels that are expected to be saturated, only if that pixel was reset in the preceding comparison time point. The scaling factor for each pixel, which is stored in the memory as a series of W bits digital combination, or as an encoded combination of said W bits, is output at real time together with the analog or digital un-scaled electrical output value of the pixel. The electrical output value of each pixel may be represented as a floating point representation, where the mantissa represents the regular value, obtained from the pixel""s analog to digital converter and the exponent representing the scaling factor.
The control circuit receives the comparison results from the decision buffer and provides the drive signal for the reset operation of each pixel. The control circuit also generates a series of reset pulses of constant predetermined frequency and duty cycle, so as to terminate the integration time. A series of sampling pulses of predetermined duty cycle and of the same frequency of the reset pulses, is generated. This series appears with a constant delay with respect to the reset pulses. The time interval between consecutive sampling pulses represents the full integration time. A series of reset enable pulses is generated, in which the time interval between consecutive pulses is shorter than the time interval between consecutive reset pulses. A control signal is generated whenever the electrical output of a pixel exceeds the threshold value. A conditional reset signal is generated whenever there is coincidence between a reset enable pulse and a control signal, and integration is started at the time of generation of either a reset pulse or a conditional reset pulse. The integration is terminated at the time of sampling pulse generation.
Preferably, the control and memory circuits are fabricated on the imager semiconductor substrate, for simplicity, speed, Signal-to-Noise Ratio (SNR) and cost reduction. The imager may comprise only one column-parallel signal chain of capacitors with associated A/D converter, which is used both for copying readouts for comparisons with threshold values, and for copying the electrical readouts of each selected row.
The invention is also directed to an optical imager with expanded dynamic range, which comprises circuitry for individually controlling the integration time of each pixel of the sensor array, and for providing a corresponding scaling factor for the electrical output of each said individual pixel during the frame time. This circuitry of the imager controls the integration time of each pixel as a function of light intensity received thereon by resetting the pixel after a predetermined threshold for the output signal has been reached.